WEAKLY MIXING GROUP ACTIONS: A BRIEF SURVEY AND AN EXAMPLE 



V. BERGELSON*, A. GORODNIK 
Dedicated to Anatole Katok on the occasion of his 60th birthday. 

1. Introduction 

At its inception in the early 1930's, ergodic theory concerned itself with continuous one-parameter 
flows of measure preserving transformations (|Bi|, |vNl|, |KvN|, |Hol|, |Ho2|). Soon it was re- 
alized that working with Z-actions rather than with M-actions, has certain advantages. On the one 
hand, while the proofs become simpler, the results for K-actions can often be easily derived from 
those for Z-actions (see, for example, |Ko|). On the other hand, dealing with Z-(or even with N-) 
actions extends the range of applications to measure preserving transformations which are not nec- 
essarily embeddable in a flow. Weakly mixing systems were introduced (under the name dynamical 
systems of continuous spectra) in |KvN|. By the time of publishing in 1937 of Hopf's book BHo3l . 
the equivalence of the following conditions (which, for convenience, we formulate for Z-actions) 
was already known. It is perhaps worth noticing that, while in most books either (i) or (ii) below is 
taken as the "official" definition of weak mixing, the original definition in |KvN| corresponds to the 
condition (vi). 

Theorem 1.1. Let T be an invertible measure-preserving transformation of a probability mea- 
sure space (X,2J,/j). Let Uj denote the operator defined on the space of measurable functions 
by (Ut f)(x) = f(Tx). The following conditions are equivalent: 

(i) For any A, B £ <B, 

^LjjL IMA n T-"B) -v(A)p.(B)\ = 0. 

(ii) For any A,B £ l B, there is a set P C N of density zero such that 

lim p.(An T-"B) = p.{A)n{B) . 

(iii) T x T is ergodic on the Cartesian square of [X, <B 

(iv) For any ergodic probability measure preserving system (Y, © , V, S), the transformation T x S 
is ergodic on X xY. 

(v) If f is a measurable function such that for some X £ C, Ujf = Xf a.e., then f — const a.e. 

(vi) For f £ L 2 (X ,« 7J u) with f x fd/j — 0, consider the representation of the positive definite 
sequence (Ujf,f),n £ Z, as a Fourier transform of a measure V onT = R/Z: 

(Urf,f) = I e 2Kim dV, n £ Z 

JT 

(this representation is guaranteed by He rglotz theorem, see |He|). Then V has no atoms. 



Remark 1.2. It is not too hard to show that condition (i) can be replaced by the following more 
general condition: 
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(i') For any A,B £ <B and any sequence of intervals /jy = [on + + 2, . . . C Z, iV > 1, 
with | /at | =bN — cin ^ °°, one has 

J™in E |/i(Anr-"B)-KAHs)| = o. 

Condition (i'), in its turn, is equivalent to a still more general condition in which the sequence of 
intervals {In}n>i is replaced by an arbitrary F0lner sequence, i.e. a sequence of finite sets Fn C Z, 
N > 1, such that for any a E Z, 

\(F N + a)DF N \ 

— : — i — ► 1 as JV — * oo. 

1^1 

This more general form of condition (i') makes sense for any (countably infinite) amenable group 
and, as we shall see below (cf. Theorem ll.6> . can be used to define the notion of weak mixing for 
actions of amenable groups. 

Remark 1.3. If is a separable space (which will be tacitly assumed from now on), the 

condition (ii) can be replaced by the following condition (see Theorem I in | KvN |): 

(ii') There exists a set P C N of density zero such that for any A,B € 2? , one has 

lim /j (A n T~ n B) = fi(A)fi(B). 

n-^><*>,n^P 

Condition (ii) in Theorem II . 1 1 indicates the subtle but significant difference between weak and 
strong mixing: while for strong mixing one has p(A n T~ n B) — > p(A)p(B) asn-> ±oo for any pair 
of measurable sets, a weakly mixing system which is not strongly mixing is characterized by the 
absence of mixing for some sets along some rarefied (i.e. having density zero) sequence of times. 
Although the first examples of weakly but not strongly mixing measure preserving transformations 
were quite complicated, numerous classes of measure preserving systems that satisfy this property 
are known by now. For instance, one can show that the so-called interval exchange transformations 
(IET) are often weakly mixing ([KS |, |V|). On the other hand, A. Katok proved in [Ka| that the IET 
are never strongly mixing. It should be also mentioned here that weakly mixing measure preserving 
transformations are "typical", whereas strongly mixing ones are not (see, for example, JH)). Before 
moving our discussion to weakly mixing actions of general groups, we would like to formulate some 
more recent results which exhibit new interesting facets of the notion of weak mixing. 

Theorem 1.4. Let T be an invertible measure-preserving transformation of a probability measure 
space {X, H ,fi). The following conditions are equivalent: 

(i) The transformation T is weakly mixing. 

(ii) Weakly independent sets are dense in <B. {Here a set A € <B is weakly independent if there 
exists a sequence n\ < n-i < • • • such that the sets T~ n 'A, i > 1, are mutually independent). 

(iii) For any A € 2? and k G N, k > 2, one has 

i AT— 1 

lim - £ p(A n T-"A n T- 2n A ("!•••("! r _ *"A) = (p(A)) k+l . 

(iv) For any t£N, k > 2, any fi,fi,---,fk € L°°(X, <B and any non-constant polynomials 
Pi{n),p2{n),...,pi c (n) ^7L\n\ such that for all j, deg(/?,— pj) > 0, one has 

jj™^ E M TPl{n)x )f2( TP2M x)---fk(T p ^ n) x) = Jfydp jf 2 dp--- Jf k dp 

in l?-norm. 
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Remark 1.5. Condition (ii) is due to U. Krengel (see |Kr| for this and related results). Condition (iii) 
plays a crucial role in Furstenberg's ergodic proof of Szemeredi's theorem on arithmetic progressions 
(see |Fj] and HF2I 1. Criterion (iv) was obtained in HBel I . Similarly to the "linear" case (iii), the 
condition (iv) (or, actually, some variations of it) plays an important role in proofs of polynomial 
extensions of Szemeredi's theorem (see |BeLl |, |BeMl |, |BeM2|, [L|). Note that the assumption 
k > 2 in (iii) and (iv) is essential. Indeed, for k = 1 condition (ii) expresses just the ergodicity of 
T, whereas for k = 1, condition (iv) is equivalent to the assertion that all non-zero powers of T are 
ergodic. The following equivalent form of condition (iv) is, however, both true and nontrivial already 
for k = 1 (cf. condition (ii') in Remark lOl : 

(iv') For any k > 1 and any nonconstant polynomials p\(n), . . . ,pk(n) S Z[n] such that for all 

i 7^ /', deg(p,- — pj) > 0, there exists a set P C N having zero density such that for any sets 

Ao,...,A£ G S, one has 

lim p(A n T PiM Ai n • • • n T Ptin) A k ) =p(A )p(A l ). . .p(A k ). 

Theorems II. 1111.41 and numerous appearances and applications of weakly mixing one-parameter 
actions in ergodic theory hint that the notion of weak mixing could be of interest and of importance 
for actions of more general groups. One wants, of course, not only to be able to come up with a 
definition (this is not too hard: for example, condition (iii) in Theorem 11.11 makes sense for any 
group action), but also to be able to have, similarly to the case of one-parameter actions, many 
diverse equivalent forms of weak mixing including those which pertain to independence and higher 
degree mixing properties of the type given in Theorem ll.4l 

Let (T g ) ge c be a measure preserving action of a locally compact group G on a probability measure 
space (X, <B ,p). If G is amenable, one can replace condition (i) in Theorem ll.lK or. rather, condition 
(i') in remark ll~2l by the assertion that the averages of the expressions \p(A<lT g B) — p(A)p(B)\ taken 
along any F0lner sequence in G converge to zero. If G is noncommutative, one also has to replace 
condition (v) by the assertion that the only finite-dimensional subrepresentation of (U g ) ge c (where 
Ug is defined by (U g f)(x) — f(T g l x), f £ L 2 (X,<B ,p)) is the restriction to the subspace of constant 
functions. H. Dye has shown in |D| that under these modifications the conditions (i), (iii), and (v) 
in Theorem 1 1.1 1 are equivalent. Dye's results are summarized in the following theorem (cf. iDl . 
Corollary 1, p. 129). Again, for the sake of notational convenience, we state the theorem for the case 
of a countable group G. 

Theorem 1.6. Let (T g ) ge o be a measure preserving action of a countable amenable group G on a 
probability measure space (X, H ,,u). Then the following conditions are equivalent: 

(i) For every F0lner sequence [F n )™ =i in G and any A,B € <B, one has 

lim TR I HAnT g B)-p(A)p(B)\=0. 

(ii) The only finite dimensional subrepresentation of (U g ) g£ G is its restriction to the space of 
constant functions. 

(iii) The diagonal action of (T g x T g ) ge <j on the product space (X x X, H ® <B ,p ® p) is ergodic 
(i.e. has no nontrivial invariant sets). 

Remark 1.7. As a matter of fact, it is not too hard to show that conditions (ii) and (iii) in Theorem 
II. 61 are equivalent for any locally compact noncompact second countable group. See, for instance, 
IIMoore l. Proposition 1, p. 157. 

A measure preserving system (X, <B ,p, T) is called a system with discrete spectrum if L 2 (X, <B ,p) 
is spanned by the eigenfunctions of the induced unitary operator Ut- It is not hard to show that the 
condition (v) in Theorem l 1 . 1 l implies that a measure preserving system (X, ® ,p, T) is weakly mixing 
if and only if it does not have a nontrivial factor which is a system with discrete spectrum. Remark 
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ll.7l hints that a natural generalization of this fact to general group actions holds as well. (A measure 
preserving action of a group G on a probability space [X 1 ® has discrete spectrum if L 2 (X, % 
is representable as a direct sum of finite-dimensional invariant subspaces.) 

In |vN2| and [H| von Neumann and Halmos have shown that an ergodic one-parameter measure 
preserving action has discrete spectrum if and only if it is conjugate to an action by rotations on 
a compact abelian group. Again, this result has a natural extension to general group actions. See 
IMacl l for details and further discussion. 

The duality between the notion of weak mixing and discrete spectrum extends to the relative 
case, namely, to the situation where one studies the properties of a system relatively to its factors. 
The theory of relative weak mixing is in the core of highly nontrivial structure theory developed by 
H. Furstenberg in the course of his proof (|F1 1) of Szemeredi theorem. See also [FK1 1 and |F2|, 
Chapter 6. 

In |Z1 1 and |Z2| the duality between weak mixing and discrete spectrum is generalized to exten- 
sions of general group actions. In particular, Zimmer established a far reaching "relative" version of 
Mackey's results on actions with discrete spectrum. 

A useful interpretation of condition (i) in Theorem ll.6l is that if (T g ) ge G is a weakly mixing action 
of an amenable group G, then for every A,B G $ and £ > 0, the set 

Ra b = {gtG: \fi(A n T g B) — /j(A)/j(B)\ < e} 

is large in the sense that it has density 1 with respect to any F0lner sequence (F„)~ =1 : 

\RDF n 



lim 



Fn 



1. 



A natural question that one is led to by this fact is whether there is a similar characterization of the 
sets Ra,b in the case when G is not necessarily amenable. 

It turns out that for every locally compact group which acts in a weakly mixing fashion on a 
probability space, the set Ra,b is always "conull", and in more than one sense. One approach, 
undertaken in | BeRo |, is to utilize the classical fact that functions of the form = fj(A n T g A) are 
positive definite. This implies that such as well as a slightly more general functions of the form 
§(g) = /j(A fl T g B), are weakly almost periodic (see |Eb|). By a theorem of Ryll-Nardzewski (see 
I R-N |), there is a unique invariant mean on the space WAP(G) of weakly almost periodic functions. 
Denoting this mean by M and assuming that for every A,B G S, the function g i— > fj(A n T g B) is 
continuous on G, let us call the action (T g ) ge c weakly mixing if for all 



f x J 2 eLl{X,<B,n) = f {feL 2 (X,$,L,): f fdfi=0}, 

Jx 



one has 



M 



( Mx)f 2 (T g x)dfi(x) 
Jx 



= 0. 



Theorem 1.8. (| BeRo |, Theorem 4.1) Let (T g ) g( zQ be a measure preserving action of a locally com- 
pact second countable group G on a probability space (X, <B The following are equivalent: 

(i) (T g ) ge c is weakly mixing. 

(ii) For every /, , f 2 G L 2 (X , <B , n), 



M 



fl(x)f 2 (T g x)dfi(x) - / fid/j / fadfi 



= 0. 



(iii) For every /o, . . . ,/„ G Lq(X, ® and £ > 0, there exists g G G with 



Mx)fi(T g x)dn(x) 



<£, ! = 1, 
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(iv) For every gi,...,g n 6 G, f G h\ (X , « , /j), anof £ > 0, f/zere exists g G G smc/i f/iaf 

f( T g x)f(T gi x)dp.(x) < e > i = l,...,n. 

(v) For aW F G L 2 (X, $ ,p), where F is not equivalent to a constant, the set {f(T g x) : g G G} is 
nof relatively compact in L 2 (X, 3? 

(vi) Lq(X, S contains no nontrivial finite dimensional invariant subspaces of (U g ) g< =G- 

(vii) (T g x T g ) ge c is ergodic. 

(viii) (r g x Tg) g £G is weakly mixing. 

We shall describe now one more approach to weak mixing for general group actions (see |Be3 1, 
Section 4, for more details and discussion). Let G be a countably infinite, not necessarily amenable 
discrete group. For the purposes of the following discussion it will be convenient to view pG, the 
Stone-Cech compactification of G, as the space of ultrafilters on G, i.e. the space of {0, l}-valued 
finitely additive probability measures on the power set <P (G) of G. Since elements of pG are {0, 1}- 
valued measures, it is natural to identify each p G pG with the set of all subsets having /^-measure 1, 
and so we shall write A G p instead of p(A) — 1. (This explains the terminology: ultrafilters are just 
maximal filters.) Given p, q G G, one defines the product p ■ q by 

A G p ■ q <^ {x : Ax~ 1 G p} G q. 

The operation defined above is nothing but convolution of measures, which, on the other hand, is an 
extension of the group operation on G. (Note that elements of G are in one-to-one correspondence 
with point masses, the so-called principal ultrafilters.) It is not hard to check that the operation 
introduced above is associative and that (pG,-) is a left topological compact semigroup (which, 
alas, is never a group for infinite G). For a comprehensive treatment of topological algebra in the 
Stone-Cech compactification, the reader is referred to | HiS | . By a theorem due to R. Ellis | El | , any 
compact semigroup with a left continuous operation has an idempotent. (There are, actually, plenty 
of them since there are 2 C disjoint compact semigroups in pG.) Idempotent ultrafilters find numerous 
applications in combinatorics (see, for example, |Hi| and |HiS|, Part 3) and also are quite useful 
in ergodic theory and topological dynamics (see, for example, |Be2|, |Be3|). Given an ultrafilter 
p G pG and a sequence {x g ) g& G m a compact Hausdorff space, one writes 

p- lim x g = y 

gEG 

if for any neighborhood U of y, one has 

{g g G : x g eU}e P . 

Note that in compact Hausdorff spaces p-limit always exists and is unique. 

The following theorem, which is an ultrafilter analogue of Theorem 1.7 from |FK2|, illustrates 
the natural connection between idempotents in pG and ergodic theory of unitary actions. 

Theorem 1.9. Let (U g ) g £G be a unitary action of a countable group G on a Hilbert space H . For 
any nonprincipal idempotent p G pG and any f G H one has 

p- lim U g f = Pf (weakly) 

where P is the orthogonal projection on the subspace 9i r of p-rigid elements, that is, the space 
defined by 

X r = {f:p-]imUgf = f}. 

Theorem ll.9l has a strong resemblance to the classical von Neumann's ergodic theorem. In both 
theorems a generalized limit of U g f, gEG, (in case of von Neumann's theorem this is the Cesaro 
limit) is equal to an orthogonal projection of / on a subspace of H . But while von Neumann's 
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theorem extends via Cesaro averages over F0lner sets to amenable groups only, Theorem 1 1 .91 holds 
for nonamenable groups as well. 

Given an element p £ pG, it is easy to see that R = p- (3G is a right ideal in PG (that is, R ■ pG C R). 
By using Zorn's lemma one can show that any right ideal contains a minimal ideal. It is also not 
hard to prove that any minimal right ideal in a compact left topological semigroup is closed (see 
[Be3 1, Theorem 2.1 and Exercise 6). Now, by Ellis' theorem, any minimal ideal in pG contains an 
idempotent. Idempotents belonging to minimal ideals are called minimal. It is minimal idempotents 
which allow one to introduce a new characterization of weak mixing for general groups. Recall that 
a set A C Z is called syndetic if it has bounded gaps and piecewise-syndetic if it is an intersection 
of a syndetic set with a union of arbitrarily long intervals. The following definition extends these 
notions to general semigroups. 

Definition 1.10. Let G be a (discrete) semigroup. 

(i) A set A C G is called syndetic if for some finite set F C G, one has 

\jAr l =G. 

teF 

(ii) A set A C G is piecewise syndetic if for some finite set F C G, the family 

[jAr^a- 1 : a£G^j 

has the finite intersection property. 

The following proposition establishes the connection between minimal idempotents and certain 
notions of largeness for subsets of G. It will be used below to give a new sense to the fact that for a 
weakly mixing action on a probability space (X, <B 7J u), the set Ra.b is large for all e > and A,B £ % . 

Theorem 1.11. (see |Be3|, Exercise 7) Let G be a discrete semigroup and p £ (PG, •) a minimal 
idempotent. Then 

(i) For any A £ p, the set B — {g : Ag^ 1 £ p} is syndetic. 

(ii) Any A £ p is piecewise syndetic. 

(iii) For any A £ p, the set 

A~ l A = {x£G: yx £Afor some y £ A} 

is syndetic. (Note that if G is a group, then A~ l A = {gi^gi ■ gi,g2 £ A}.) 

Definition 1.12. A set A C G is called central if there exists a minimal idempotent p £ pG such that 
A £ p. A set A C G is called a C* -set ( or central* set) if A is a member of any minimal idempotent in 
PG 

Remark 1.13. The original definition of central sets (in Z), which is due to Furstenberg (see |F2|, 
p. 161), was the following: a subset S C N is a central set if there exists a system (X,T), a point 
x £ X, a uniformly recurrent point y proximal to x, and a neighborhood U y of y such that S = {n : 
T"x £ Uy}. The fact that central sets can be equivalently defined as members of minimal idempotents 
was established in |BeH|. See also Theorem 3.6 in IBe3l. 

The following theorem gives yet another characterization of the notion of weak mixing. 

Theorem 1.14. (see |Be3|, Section 4) Let (T g ) ge G be a measure preserving action of a countable 
group G on a probability space (X, $ ;J u). Then the following are equivalent: 

(i) (T g ) ge Q is weakly mixing. 

(ii) For every f £ L 2 (X,'S and any minimal idempotent p £ pG, one has 




fdp (weakly). 
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(iii) There exists a minimal idempotent p £ pG such that for any f £ L 2 (X ,$ one has 
p-]im geG f(T g x) = f x fdp (weakly). 

(iv) For any A^B £ <B and any E > 0, the set 

{g£G:\p(AnT g B)-p(A)p(B)\<e} 

is a C* -set. 

Given a weakly mixing action of, say, a countable (but not necessarily amenable) group G, one 
would like to know whether the action has higher order mixing properties along some massive and/or 
well-organized subsets of G. For example, it is not hard to show that for any weakly mixing Z-action 
and any nonconstant polynomial p(n) £ Z[n], one can find an IP-set S such that for any A,B £ <B, 
one has 

lim p{AC\T p{n) B)=p{A)p(B). 

n— >°o,«GS 

(An IP-set generated by a sequence {n, : i > 1} is, by definition, any set of the form {«;, H h nt k : 

/!<■■■< i k ; k £ N}.) 

Another example of higher degree mixing along structured sets is provided by a theorem proved 
in |BeRu|, according to which any weakly mixing action of a countable infinite direct sum G = 
©„>iZ p , where Z p is the field of residues modulo p, has the property that the restriction of the 
action of G to an infinite subgroup (which is isomorphic to G) is Bernoulli (see also |BeKMl|, 
HBeKM2l . IIBeKLMI . IJRWl . Q, ifrfl V 

In Section 2 below we give a detailed analysis of higher order mixing properties for a concrete 
classical example — the standard action of SL(2,Z) on the 2-dimensional torus T 2 . Since SL(2,Z) 
contains mixing automorphisms (namely, hyperbolic automorphisms), this action is weakly mixing. 
On the other hand, this action is not strongly mixing because SL(2, Z) contains nontrivial unipotent 
elements. 

While many of the results obtained below hold (sometimes, after an appropriate modification) 
for toral actions of SL(n,Z) and even in more general situations, we intentionally deal here with 
SL(2, Z) -actions in order to make the paper more accessible and important issues more transparent. 

Here is a sample of what is proved in the next section: 

• (cf. Proposition l2.10> Let 7i , , 7^ £ SL(2,Z). Then the following assertions are equiva- 
lent: 

(i) For every Aq, . . . ,Ak £ ® , 

p(A n T"A\ n • • • n T k n A k ) p(A ) ■ ■ -p(A k ) as n -f oo. 

(ii) Each 7} is hyperbolic, 7) ^ ±7) for i ^ j, and for every p > 1, there are at most two 
matrices among 7}, i = l,...,k, having an eigenvalue A- such that |A.| = p. 

• (cf. Proposition 12. 20> Let 7\ , , 7]t £ SL(2,Z) be hyperbolic automorphisms. Denote by 

A,,- the eigenvalue of 7/ such that |A,,| > 1. Put ao, n = 0, n > 1. Let k > 1 and a,,„ £ Z, 
; = 1 be such that 

min{|log \Xj\ ■a^„ — log \Xj\ -aj M \ : < i < j < n} — > °° as n^°°. 

Then for every Ao, . . . ,A k £ <B , 

p(A r\T" L "A l n---r\T^"A k )^p(A )---p(A k ) as n^°o. 

This result generalizes Rokhlin's theorem |R| in the case of 2-dimensional torus. See also 
Proposition ^. 24l for an analogue of this result for unipotent automorphisms. 

• While every abelian group of automorphisms G, which acts in a mixing fashion on T 2 , 
is mixing of order k for every k > 1, (that is, for every k > 1 and sequences go „ = e, 
gt,n,---,gk,n£G such that 

<?;><?./> — ► °° as n -> °o for < i < j < k, 
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one has 

v(AoC)g Un A l r\---r\g k _„A k )^ij(Ao)---v(Ak) as n -> «.) 

a nonabelian group of automorphisms of T 2 is never mixing of order 2 (see Proposition 
I2.3H . Note that there are nonabelian groups of automorphisms that act in a mixing fashion 
on T 2 (see the discussion after Proposition l2.30> . 

2. SL(2,Z)-ACTION ON TORUS 

Definition 2.1. A sequence T„ G SL(2,Z), n> 1, is called mixing if for every /i,/2 € L°°(T 2 ), 



(1) j^Mmh{%)d%^ [J r2 M^ 

A transformation T G SL(2,Z) is called mixing if the sequence T", n > 1, is mixing. 
Note that this definition is different from the one given in |BBe |. 

Recall that a matrix T is called hyperbolic if its eigenvalues have absolute values different from 
1, and unipotent if all its eigenvalues are equal to 1. It is well-known that an automorphism T G 
SL(2,Z) is mixing on the torus T 2 if and only if it is hyperbolic. This implies that the action 
of SL(2,Z) on T 2 is weakly but not strongly mixing and motivates the following problem: give 
necessary and sufficient conditions for a sequence T„ G SL(2,Z), n > 1, to be mixing. 

We start with a useful and straightforward lemma (cf. Theorem 3.1(1) in IB2I - ). For a matrix T, 
denote by 'T its transpose. 

Lemma 2.2. A sequence T„ G SL(2,Z), n > 1, is mixing if and only if for every (x,y) G (Z 2 ) 2 — 
{(0,0)}, the equality 'T n x +y = holds for finitely many n only. 

Proof. To prove that T n is mixing, it is sufficient to check Q for f\ and fa in the dense subspace of 
trigonometric polynomials. It follows that T n is mixing if and only if Q holds for f\ and fa that are 
characters of the form 

(2) %M = e 2ni{xA) , xeZ 2 £eT 2 . 

Forx,y G Z 2 , one has 

2 x.wfo t ®dt= [ x w (^=(i ^ yi^n 

t 2 Jt 2 " ■ ii if /;,v • v o. 

It follows that for (x,y) G (Z 2 ) 2 - {(0,0)}, 

1 X,(r»^(^^(/ a Xx(^)(j^ a X,(^)=0 as n^oc 
if and only if the equality 'r„x + y = holds for finitely many n only. This proves the lemma. □ 

Denote by M(2,K) the set of 2 x 2-matrices over a field K. Using Lemma l2~2l we can now prove 
the following proposition. 

Proposition 2.3. Let T„ e SL(2,Z), n > 1, || • || be the max-norm on M(2,M), and D C M(2,E) 
denote the set of limit points of the sequence p?| as n — > oo. Then the sequence T„ is not mixing 
if and only if there exist A S M(2,Q) and B G M(2,Q) such that B G 23 ant/ r„ = A + \\T„\\B for 
infinitely many n > 1. 

Proof. We may assume that ||7],|| °°. (Indeed, if j|7J, | oo, then there exists a matrix 7b such that 
r„ = To for infinitely many «, and the statement is obvious.) 
"<=": Let T n =A + \\T n \\B. Since 

detB = lim det [ ] = lim -^-^ = 0, 



WEAKLY MIXING GROUP ACTIONS: A BRIEF SURVEY AND AN EXAMPLE 



1 1 



B is degenerate. Thus, there exists x 6 Z 2 — {0} such that 'Bx = 0. Then for infinitely many n, 
'T„x = 'Ax, and, by Lemma l2~2l T„ is not mixing. 

"=►": By Lemma l2~2l there exists (x,y) S (Z 2 ) 2 — {(0,0)} such that 'T„x = —y for infinitely many 
n. By passing, if needed, to a subsequence, we may assume that this equality holds for all n > 1. It is 
clear that gcd(jti ,x 2 ) = gcd(yi ,^2)- Thus, we may assume that x and y are primitive (that is, the gcd 
of their coordinates is 1). Take C,D £ SL(2,Z) such that Ce\ —x and Dei = — y where e\ = (1,0). 
Then 



%=D 



1 a„ 
1 



C _1 = DC 1 



for some a„ € Z. Put % = DC 1 and f F 2 = D 

T n =Fi+ a„F 2 



1 




n 1 

a " D {o 

C _ 1 . We have 



C 



-1 



(3) 
and 
(4) 

Hence, ||r„| 



|fln|-||F 2 ||-||Fi|| < ||r„|| < |a„|-||F 2 || + ||Fi||. 
\u n \ ' \\F2W as n — > 00. Replacing, if necessary, F2 by — F 2 and a„ by — a„ we may 



assume that a„ > for infinitely many n. Then B = e 2?. Passing to a subsequence, we get 
that a„ > for n > 1. By triangle inequality and @, 



F 2 



ll*2|| 



F 2 



ll^|| 



1*1 I 



|r„||| <2||Fi| 



Thus, for infinitely many n, T n — \\T n \\B =A for some A <G M(2,Q). This proves the proposition. □ 

We illustrate the usefulness of Proposition l2.3l bv the following two propositions. 

Proposition 2.4. Let U,V £ SL(2,Z) fee unipotent matrices. Then the sequence T„ = U~"V n is 
mixing if and only ifUV ^ VU. 

Proof. If U and V commute, one can show that they are powers of a single unipotent transformation. 
Hence, in this case, the sequence T n = U~"V" is not mixing. 

Conversely, suppose that UV ^ VU. There exist A, B 6 SL(2,Z) such that 



U=A~ 



1 u 
1 



A and V = B 



-1 



1 v 
1 



B 



for some b,v£Z 
a fe 



AS" 1 = 

5« = 
When c = 0, 



{0}. It is sufficient to show that the sequence S„ = AT n B 1 is mixing. Let 
We have 



—nu 
1 



AB 



-1 



1 nv 
1 



a — (cu)n b - 
c 



(av + c/m)« — (cv)n 2 
d + (cv)n 



V =B 



-1 



B = B- l {AB~ l ) 



1 \ 1 



(AB-')B=A 



-1 



A. 



and it follows that U and V commute. Thus, c 7^ 0. 

We apply now Proposition l2.3l For sufficiently large n, ||S„|| = \b— (av + du)n — (cv)n 2 \. Also 



-sign(cv) 




def 



C. 



Since S n — ||5„||C is not constant for infinitely many n, the sequence S„ is mixing. 



□ 



Remark 2.5. When U, V E SL(2, Z) are commuting unipotent transformations, the sequence U "V" 
is relatively mixing in the sense of Definition l2.22l below. 
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Using a similar argument, one proves the following proposition: 

Proposition 2.6. Let U,V G SL(2,Z) such that U is unipotent, and V is hyperbolic. Then the 
sequence T n = U~"V" is mixing. 

Proof. Denote by the 2x2 matrix with 1 in position (i.j) and O's elsewhere. For some A.B E 
GL(2,R) and A, with \X\ > 1, 

tf-A-(J ,°,)a and K-B"(J | IB. 

We write 
where 



r„=A-'( Q JAB - I * [ )B = X- n C + X-"nD + X n E + X n nF 



C=A~ 1 E n A, D=A- 1 E n AB- 1 Ei 2 B, E=A~ 1 E 2 2A, F = A~ 1 E22AB~ l EnB. 
Suppose that F =/= 0. Then — > pjj. By Proposition 12.31 we need to show that there is no 
X G M(2,K) such that T n — \\T n \\ py = X for infinitely many n. Since F is degenerate, one of the 
matrices C, D, E is not a scalar multiple of F (say C). Take a basis of M(2,R) which contains C 
and F. The C-coordinate of T„ — || T n \\ prr with respect to this basis is equal to X~ n + aX~"n + $X" 

for some a, (3 G R. This shows that the sequence r„ — ||r„||p^ consists of distinct matrices for 
sufficiently large n. Thus, T„ is mixing. 

Suppose that F = 0. Then p?^ — > pp By the same argument as in the previous paragraph, the 

sequence T n — ||2n||pjy consists of distinct matrices for sufficiently large n. This implies that T„ is 
mixing. □ 

Remark 2.7. When U,V G SL(2,Z) are hyperbolic and U ^ V, the sequence U~ n V n is mixing. 
This follows from Proposition l2.10l below. 

Next, we study multiple mixing for general sequences. 

Definition 2.8. Let Ti^ n G SL(2,Z), n > 1, i=l,. . . ,k. The sequences T\ ni . . . ,T^ t „ are jointly mixing 
if for every f £ L°°(T 2 ), i— l,,,,,k+l, 

(5) f^MT 1 £y--MTk*m+i®<*^ (/ t2 /i(^) •■■ (Lfk+i(S)i% 

as n — > oo. Transformations 7i,...,7j are called jointly mixing if the sequences T", . . . ,77, « > 1, 
are jointly mixing. 

In |B2 1, this property was called w-jointly strongly mixing (see Definition 3.6 in |B2|). 
In the course of proving Proposition ^. lOl below. we shall need the following immediate extension 
ofLemmal2~2l(cf. Theorem 4.3(1) in lEH l. 

Lemma 2.9. Let 7} „ G SL(2,Z), n> I, . . ,k. The sequences 7j „, . . . ,7^ jn are jointly mixing if 
and only if for every (x\,. . . G (Z 2 )* +1 — {(0, ... ,0)} //ze equality 

r Ti, n x\ H \-%, n Xk +x k+ i=0 

holds for finitely many n only. 

Proposition 2.10. Lef 7; £ SL(2, Z), / = 1 , . . . , k. The transformations T\ , . . . , are jointly mixing 
if and only if each ofTj is hyperbolic, Tj ^ ±7) for i ^ j, and for every p > 1, there are at most two 
matrices among T, i — I,... ,k, having an eigenvalue X such that \X\ = p. 
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Proof. If a matrix 7 £ SL(2,Z) has complex eigenvalues, they are units in an imaginary quadratic 
field. This implies that the eigenvalues of 7 are roots of unity. Hence, the transformation 7 is not 
mixing on T 2 . Therefore, we may assume that the eigenvalues of 7 are real. 

Next, we note that one can assume without loss of generality that the eigenvalues of 2J, i = 1, . . . , k, 
are positive. Indeed, put f, ■ = — Tj if the eigenvalues of Tj are negative and 7; otherwise. Clearly, 
transformations 7], i = l,...,k, are jointly mixing if and only if transformations f[, i = l,...,k, are 
jointly mixing. 

Let the transformations T\, . . . , be jointly mixing. Then each of the sequences T" and T^"T", 
i 7^ /', is mixing too. This implies that all 7; are hyperbolic and 7] ^ Tj for i ^ j. To show that the 
conditions of the theorem are necessary, we consider transformations 7i, 7*2, 7*3 £ SL(2,Z) that have 
the same eigenvalue A, > 1. We claim that there exists (xi,x 2 ,x 3 ) £ (Z 2 ) 3 — {(0,0,0)} such that 

(6) ^1+^2 + ^3=0 

for every n > 1, which, in view of Lemma l2~9l implies that the sequences T", 7 2 ", 7 3 " are not jointly 
mixing. 

Since 7), i = 1,2,3, have the same eigenvalues, there exist A, B £ GL(2,R) such that 

(7) T 2 =A' l T l A and T 3 =B l TiB. 
Note that the matrix A is a solution of the matrix equation 

(8) XT 2 = T X X, 

which can be rewritten as a homogeneous system of linear equations with rational coefficients. The 
set of rational solutions of (|8} is dense in the space of real solutions. It follows that there exists a 
rational solution l|8} such that det(X) ^ 0. This shows that we may choose A and B in GL(2,Q). 
For every v £ R 2 , v — v + + v_ where v + and v_ are eigenvectors of l T\ with eigenvalues X and Ar 1 
respectively (X > 1). Define linear maps P + :fhv + and P : v i— > v_. Then 

(9) % = 1P+ +X~ 1 P-, P+P =P-P+=Q, P±=P±, P++P-= id. 

Note that P+,P £ M(2,Q(y/d)) for some d £ N determined by X. When \fd £ Q, X and X' 1 are 
algebraic integers in Q, and it follows that that X = ±1, which is a contradiction. Thus, \fd ^ Q. 
Denote by o the nontrivial Galois automorphism of the field extension Q(vrf) /Q. Then A° = X~ l 
and (P+)° = P . Using and (|9}, we may rewrite equation as 

X" (P+xi +'AP + r A- 1 x 2 + 'BP + t B- l x 3 ) +X-" +'APJA- 1 x 2 + 'BPJB- 1 x 3 ) = 0. 

The columns of the matrices P + , A7+A -1 , and t BP + 'B^ i lie in the vector space Q(^/d) 2 that 
has dimension 4 over Q. Thus, these columns are linearly dependent over Q, and there exists 
{xi,X2,X3) £ (Z 2 ) 3 - {(0,0,0)} such that 

P+xi + 'AP+'A- l x 2 + 'BP+'B- l x 3 = 0. 

Applying o to this equality, we get 

TLjq + 'APJA~ l x 2 + 'BPJB~ l x 3 =0. 

This implies © and proves that the conditions in the proposition are necessary for mixing. 

To prove sufficiency consider Si,Tj £ SL(2,Z), i = l,...,k, such that 5, and 7] have the same 
eigenvalue Xj > 1, and < Xj for i < j. We need to show that the transformations S\,T\, , . , ,Sk, 7* 
are jointly mixing. By Lemma l2~9l it is enough to prove that there is no (xi,yi,... ) Xj c ,yj e ,z) £ 
(Z 2 ) 2A+1 - {(0, ... ,0)} such that the equality 



(10) 



'SJjci + % n yi + ■■■ + 'S" k x k + %"y k +z = 0. 
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holds for infinitely many n > 1 . Suppose that such a (2k + 1 )-tuple exists. Without loss of generality, 
we may assume thaty^ ^ 0. As above, we define Pt,±,Qi,± <G M(2,R) such that 

% = X i P it+ +Xr 1 P i> _, P it+ P it _=P it _P ii+ = 0, Pt ± =P,,±, P i& + P it -= id, 
% = hQi^+X^Qi^, Q,,+Q,,- = Qi,-Qi,+ = 0, Ql ± = Q u± , Q u+ + Q u _ = id. 
Then fl!0l > can be rewritten as 

k k 

£ V fa-* + 2'' + E x ? + G^+y) + z = °- 
i=i i=i 

Dividing this equality by X" k and taking a limit over a subsequence rij — > oo, we deduce that 

Pk,+xk + Qk,+yk = 0. 

Suppose that Qk+yk — 0- Then y k ^ is a rational eigenvector of with eigenvalue A^T . It follows 
that A,t, A^T 1 € Q. On the other hand, X k and A^T 1 are algebraic integers. Hence, X k = ±1, which is a 
contradiction. This shows that 

clef 

v = = -Qk,+yk ^ 0- 

We have 

S^v = X^v = T k v. 

As above, we denote by Ok the nontrivial automorphism of the quadratic extension Q(Ajt) /<Q>. Then 

fg. = ft,-, Ql k , + = Qk.-, K k = K\ 

and it follows that 

S k v a "=X^v^=T k v a K 

Since v and v a * are linearly independent, this implies that S k = T k , which is a contradiction. Thus, 
l llOt holds for finitely many n only. The proposition is proved. □ 



Proposition ^. lOl shows. in particular, that transformations T\,Ti,Ts£ SL(2,Z) need not be jointly 
mixing even when every two of them are jointly mixing. Nonetheless, pairwise conditions are suffi- 
cient to imply mixing in the commutative situation. 

Proposition 2.11. Let T\,...,T k € SL(2,Z) be commuting automorphisms off 2 . Then they are 
jointly mixing if and only if the transformations TJ and Tf 1 Tj, i ^= j, are mixing. 

Proof. It is clear that if T\ , . . . , T k are jointly mixing, then 7j and Tf 1 Tj, ; ^ j, are mixing. 

Conversely, suppose that 7} and T^ l Tj, i ^ j, are mixing. Then 7; ^ ±7) for i =/= j. Since 7} and 
Tj commute and are hyperbolic, they can be simultaneously reduced to the diagonal form. Thus, 
if TJ and 7) have the same eigenvalues of the same modulus, then 7] = ±T- . It follows that the 
conditions of Proposition ^. 101 are satisfied and hence, Tj , . . . , 7^ are jointly mixing. □ 



One can show that the natural analog of Proposition ^. 1 ll holds in every dimension. 

In the case of a single measure preserving transformation T, T is mixing if and only if for every 
k > 1, the transformation T k is mixing. In the following proposition, we investigate what happens 
for general sequences in our group: 

Proposition 2.12. Let T n G SL(2,Z), n > 1, be hyperbolic automorphisms. Let X n be the eigenvalue 
ofT n with | An | > 1. 

(1) For any k > 1, if the sequence X„ is bounded, then T n is mixing if and only ifT„ is mixing. 

(2) For any k>2, ifX n — > °° the sequence is always mixing. 



WEAKLY MIXING GROUP ACTIONS: A BRIEF SURVEY AND AN EXAMPLE 



15 



Proof. Let t„ = Trace(r„). Then T n is a root of its characteristic polynomial x 2 — t„x + 1 . Using the 
polynomial identity: 

x k = P(x) (x 2 - t n x+ 1) + a„, k x + p\ fc 

where a„, k ,fi„. k £ Z, 

\k _^—k J— k+\ _ life— 1 



— « n D 

*"n,k « - _ i i Mh 



we deduce that 

(11) T* = On tk T n + % ik . 

Suppose that X n , n > I, is bounded. Then the sequences a„.k and $„ k are bounded, hence take on 
only finitely many values. Hence, the equality 

(12) % k x + y = %(On,kx) + ($ n ,kX+y) = 

holds for some (x,y) £ (Z 2 ) 2 - {(0,0)} and infinitely many n if and only if the equality 'r„x' +y' — 
holds for some (*',/) £ (Z 2 ) 2 - {(0,0)} and infinitely many n. By Lemma l2~2l this proves the first 
part of the proposition. 

We assume now that X„ — > °°. Then 

On^A* -1 and $ n>k X^ 2 as n->°o. 

By Lemma l2~2l it is sufficient to show that if (I12> holds for infinitely many n, then x = y = 0. Suppose 
that \\2\ holds for infinitely many n. Dividing by (X„.k and taking a limit over a subsequence rij — ► °°, 
we conclude that T nj x — > 0. Since the sequence T^x is discrete, it follows that x = 0, and y = 0. 
Thus, T„ is mixing. □ 

Remark 2.13. Note that the statement in part (2) of Proposition 12. 12l fails for k = \. For example, 
let 

t„ = y " " i 1 J ' " - 1; 

If X„ denotes the largest eigenvalue of T„, then clearly, X„ — > «>. However, the sequence T„, n> 1, is 
not mixing. (This follows from Proposition l2.3l ') 

Recall a theorem of Rokhlin | R | : 

Theorem 2.14. (Rokhlin) Let T be a mixing automorphism of a compact abelian group. Then the 
sequences T a, M , . . . , T ak -" are jointly mixing provided that 

min{|a lin — a/, n | : < z < j < n} — » °° os n — ► °o, 

where ao.„ = 0. 

The following proposition shows that a naive generalization of Rokhlin's theorem to a general 
sequence of automorphisms T„ is false. 

Proposition 2.15. Let T n £ SL(2,Z), n > 1. Denote by A,,, the eigenvalue ofT n such \k n \ > 1. If the 

sequence X n , n > 1, k bounded, then for any choice of at € Z, z' = 1, . . . (k > I) the sequences 
T" [ , . . . , T n k are not jointly mixing. 

Proof. Without loss of generality, we may assume that a,- > 0, i = 1 , . . . , k. 

By Lemma l2~9l it sufficient to show that there exists a tuple (x\,... ,x k+ \) £ (Z 2 ) k+1 — {(0, ... ,0)} 
such that for infinitely many n, 

T«x 1 + — + , T?x k +x M =0. 
By dl2> . the last equality reduces to 

(13) 'T n {a n . ai xi H ^a n .a k x k ) + % Ml xi H 1- $n,a k Xk +x k +\ =0. 
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Since the sequence X n , n > 1, is bounded, the sequences a nX!j and p„. fl , are bounded too. Thus, 
they are constant for infinitely many n. Now one can easily choose Xi € Z 2 , i = l,...,k+ 1, not all 
zero, such that Jl 3i holds. For example, one can take all x,'s to be multiples a fixed nonzero integer 
vector. □ 

Remark 2.16. Even if a sequence T n E SL(2,Z), n > 1, is such that 

(i) T n is hyperbolic and mixing on T 2 for all 

(ii) X„ — > °o, where X„ is the eigenvalue of T n such that A,,, > 1, 
the sequences T n and T t f need not be jointly mixing. For example, put 

Then 'r„ 2 x + 'T n y + z = for x = r (0, 1), y = '(—2,0), z — '(0, —2) which implies that the sequences T n 
and T„ 2 are not jointly mixing. On the other hand, it follows from Proposition l2.3l that the sequence 
T n is mixing. This example also demonstrates that pairwise conditions are not sufficient to guarantee 
joint mixing even when the elements commute for every fixed n. 

We give here a generalization of Rokhlin's theorem in the case of the 2-dimensional torus. (A 
similar extension of Rokhlin's theorem holds in any dimension.) 

Proposition 2.17. Let T„ £ SL(2,Z), n > 1. Denote by X„ the eigenvalue ofT n such that \X n \ > 1. 
Put oo,n = 0, « > 1. Let k>\ and a ln 6Z, /=l,...,fc Denote 

y„ = mm{\a ifl ~a^ n \ :0<i<j <n). 

Suppose that one of the following conditions holds: 
(1) The sequence T n is mixing, and 



(2) 



0> 



n > 1 > is bounded. 



■ as 



Then the sequences t" l " , . . . , T" kM are jointly mixing. 

Remark 2.18. Part (2) of the theorem with T„ = T, n> 1, implies Rokhlin's theorem for the case 
of 2-dimensional torus. 

Proof. Since T„ is measure-preserving, we are allowed to replace a,- „ by a,- „ — min{a,„ : i = 0, . . . ,k}. 
It follows that without loss of generality, we may assume that 

min{a,.„ : i = 0, . . . ,k} =0. 

Also by changing order and passing, if needed, to subsequences, we may assume that 

max{fl,'„ : i = 1, . . . ,k} = ak, n - 

Suppose that the sequences T," u " , . . . ,T n k ' n are not jointly mixing. By Lemma l2~9l there exists a 
tuple (jci , ,xu+\) e (Z 2 ) k+l - {(0, . . . ,0)} such that the equality 

T„ xi H VT n x k +x k+i =0 

holds for infinitely many n. By (lilt , the last equality is equivalent to 

k k 

52 a n,a,jT n Xi + £ P„, a , , X; +Xk+l = 0. 
(=1 ;=1 
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Note that in both cases, iCr? — > °° as n — > °°. Therefore, it follows that K n 

y a i.n -\ a i,n 1 



a n,k~ a n,i 



Then 



k ~ l a k R 



(=1 a ",«t.„ ,'=1 a «,a*. n a n,«*,„ 

Assume that condition (1) holds. Then the sequence f T n Xk is bounded by infinitely many n. Thus, 
it is constant for infinitely many n. It follows from Lemma l2~2l that jtj = 0. 

Suppose that condition (2) holds. We prove that x^ = 0. If X„ — » °°, then 'T n Xk — > as n — > oo, and 
this implies that jc* = 0. Otherwise, the sequences A,„ and 'T n Xk are bounded for infinitely many «, 
and consequently, they are constant for infinitely many n. Thus, 'TnjX^ = \~ Xk for a subsequence 
itj, and if ^ 7^ 0, then X nj ,X^ £ Q. Since A,,^. is an algebraic integer, X n , = ±1, which contradicts 
condition (2). This shows that jc* = 0. 

Now the proof can be completed by induction on k. □ 

Remark 2.19. Condition (1) is not necessary for joint mixing of the sequences T n ' , . . . , T n . For 
example, put 

( n 1 n 3 -l \ 
T„ = I , , n > 1, and a, „ = i, i= 1,2. 

V 1 n ) 

Even though — > °° as n — > °o, one can check with the help of Lemma l2~9l that the sequences T„ 
and r„ 2 are jointly mixing. It would be of interest to find a necessary and sufficient condition for 
joint mixing of sequences of the form T„ h " , . . . , Tn k ' n . 

The following proposition is yet another generalization of Rokhlin's theorem. 

Proposition 2.20. Let T\ , . . . , 7# G SL(2, Z) be hyperbolic automorphisms. Denote by A,,- the eigen- 
value ofTi such that |A,, | > 1. Put oq „ = 0, n > 1. Let k > I and ai „ £ Z, i = 1, . . . ,k be such that 

(14) min{|log|A,;| •«(> — log \Xj\ -aj M \ : <i< j < n} — > °° as n^>°°. 
Then the sequences T" [ " , . . . , 7^°*'" are jointly mixing. 

Proof. As in the proof of Proposition ^. 171 we reduce the proof to the case when 
log \X i+ i I -ai+^n- log |A,j| •«;,„—>• 00 as « -> °= 

for z' = 0, . . .,k— 1. 

Suppose that the sequences Tf 1 '", . . . ,Tj* k '* are not jointly mixing. By Lemma l2~9l there exists 
(xi, . . . ,Xk,y) € (Z 2 ) k+l - {(0, . . .,0)} such that the equality 

(15) t T" ij -x l + ---+X' k -" Xk +y = 

holds for infinitely many n. Let P, \+,P, > - € M(2,M), z = 1, ... ,k, be such that 

T^Vi^+V 1 ^,-. ^-=^+=0, p t±= p i,±i Pi,+ + Pi -= id. 

By 0, 

(16) £ x?' n p i>+ xi + 1 K ai '"Pi,-*i +y = o 

i=l i=l 
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holds by infinitely many n. Dividing by X k k ' n and taking limit over a subsequence nj — > °°, we 
conclude that Pk,+Xk = 0. 

If Xk ^ 0, it is an eigenvector of 'Tk with the eigenvalue A.^ 1 . This implies that X k G Q. On the 
other hand, A* is an algebraic integer. Thus, Xk = ±1. This contradiction shows that x* = 0. Using 
induction on k, we deduce from dl6> that x, = for i = 1 , . . . , k. This shows that the sequences 
T" 1 '" t£** are jointly mixing. □ 

Remark 2.21. It clear that condition dl4t in Proposition ^. 20l follows from the following condition: 

a\,n — ► °° and : > °°, i — l,...,k, as n— 

which also appears in Proposition ^. 241 

Definition 2.22. Let T t G SL(2,Z), i = Denote by P t : L 2 (T 2 ) -> L 2 (T 2 ), z = 1, . . . f/ze 

orthogonal projection on the subspace of Ti-invariant functions. Let a^ n € Z, z" = 1, n > 1. We 
call the sequences T" 1 '" , ... ,T? k ' n relatively jointly mixing if for every f GL°°(T 2 ), z'= 

(17) [ h{T^)...f k {T^ k %)f k+l ^)d^ [ (P,h){%)---{P k m%)fM{\)d% as n^oo. 

We have the following criterion for relative joint mixing of tuples of unipotent elements: 

Proposition 2.23. Let 7] G SL(2,Z), z' = 1 , . . . ,k, be unipotent elements. Denote by v,-, i = 1, . . . ,k, a 

nonzero vector such that f T{Vi — Vj. Let ai„ GZ, i=l,...,k, n > 1. Then the sequences T\ T k k ' n 

are relatively jointly mixing if and only if for every (oci , . . . , a k ) G Z* - { (0, . . . , 0)} andz G Z 2 - {0}, 
the equality 

k 

(18) £ a,-a,- n \'i + z = 

/=i 

holds for finitely many n only. 

Proof. For some A,- G SL(2, Z) and Sj G Z - {0}, 

(19) = AT 1 ( J ) Aj = £ + Si^B, 

where B; = Ar 1 f q q ^ SL(2, Z), and £ is the identity matrix. To establish relative mixing, 

it is sufficient to check dl7l in the case when i — l,...,k, are characters of the form (0. For 
X\,... ,x k+ \ G Z 2 , one has 

/ iJ x„(r;"5)-x„(rr%, t ,(5« = ffyp^f^fi* 

•+'7^ =0, 



Note that for every x G Z , 




if B iX ^ 0. 

Thus, d!7i always holds for f = % x , provided that = for all i = 1, . . . ,k. It follows that the 

sequences t" 1 " ,...,T k k ' n are relatively jointly mixing if and only if for every (x\ , . . . ,x k ) G (Z 2 )* 
such that for some z = 1 , . . . , k, BjXj ^ (equivalently, Tpc,- ^ x,) the equality 

(20) r r;"'"x 1 + -..+'r; A "x i +x, +1 =o 
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holds for finitely many n only. By (I19> . the last equality is equivalent to 

J^Sia^BiXi+z = 

/=i 

where z = Y!j=i x i- Note that the columns of the matrix are rational multiples of the vector v,. 
Thus, SjBiXi = a,v,- for some a; G Q. Since BjXj ^ for some i, (oci ,...,aj c ) ^ (0, ... ,0). This shows 
that J20> holds if and only if 

k 

f=l 

for some (ai , ...,(Xfc) G Q — {(0, . . . ,0)} and zeZ. Multiplying by a fixed integer, we get that 
a,- G Z. This proves the proposition. □ 



We record here a convenient corollary of Proposition ^. 231 

Proposition 2.24. Let Tj G SL(2,Z), i = l,...,k, be unipotent elements, and a l n G Z, / = 1, . . . ,/c, 
n > 1, such that 

0\,n — ► °° a™ ■ ► °o, I=1,...,JC, OS « — > oo. 

r/zen f/ze sequences T^' n , . . . , 7^ are relatively jointly mixing. 

Proof. Suppose that the sequences t" 1 '", T k k ' n are not relatively jointly mixing. Then by Propo- 
sition |^23] Jl 81 holds for infinitely many n. Dividing Jl 81 by ak : n and taking the limit over a 
subsequence n s —> °°, we deduce that = 0. Similarly, it follows that a,- = for i = 1 . . . . , k. This 
shows that the sequences T" 1 '" T. k: " are relatively jointly mixing. □ 

Let T.S G SL(d,Z). It was observed by Boshernitzan that it follows from the fact that the set of 
common periodic points of T and S is dense in T d that for every nonempty open subset 11 of T d , 

ur\T"u ns"« 7^0 

for infinitely many n. A measurable analogue of this fact is far less trivial. The following conjecture 
seems plausible: 

Conjecture 2.25. Let T,S G SL(rf,Z), and let <D be a Borel subset of T d of positive measure. Then 

limsup p((Dr\T"'DnS"'D) >0. 

In fact, in all known to us examples, 

limsup i u(©nr"©n5"©) >^(®) 3 . 

Remark 2.26. Note that when T and S generate a (virtually) nilpotent group, Coni ecture 12 . 251 fol- 
lows from a general "nilpotent" multiple recurrence theorem proved in ]E| (see also Theorem E in 
HBeL2l ~). It was, however, proved in |BeL3 1, that for any finitely generated solvable group of expo- 
nential growth G, there exist a measure preserving action (T g ) ge G on a probability space (X, <B 
elements gi,g2 G G and a set D G ® with fi(D) > such that for T = T gl and S = T g2 , one has 
fi(Df] T"D n S"D) = for all n ^ 0. Nevertheless, we believe that for our special action of SL(d, Z) 
on T d , the Conjecture is true. 

We obtain below some partial results on the conjecture in the case of the 2-dimensional torus. 
Note that when T and S are hyperbolic the conjecture follows from Proposition ^. 101 In fact, in this 
case, 

U©) 2 if T=S, 



lim i u(Dnr"©n5"©) = 



n{vy if t^±s 
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and when T = — S, the limit set of the sequence ^(2)fir"©nS"f) consists of two numbers: p.('D) 2 , 
nifD H — In particular, this shows that liminf might be even when /u(2>) > 0. 

We can also settle the case when T and S are unipotent and hyperbolic respectively. For this, we 
need a lemma: 

Lemma 2.27. Let T € SL(2,Z) fee unipotent, and S € SL(2,Z) hyperbolic. Then the sequences T" 
and S n , n > 1, are relatively jointly mixing. 

Proof. As in the proof of Proposition l2.23l it is sufficient to show that for every x,y,z € Z 2 such that 
either Tx ^ x or y ^ 0, the equality 



holds for finitely many n only. We have 

<T n =E + nB 

where E is the identity matrix and B G M(2,Z). Let A, be the eigenvalue of S such that \X\ > 1. For 

someP + ,P_ e M(2,R), 

'S n = X"P+ + lT n P- , P+P = P P+ = 0, Pi = P±, P+ + P = id. 
Equality d2H is equivalent to 

+ lT"P-y + nBx+{x + z) = 0. 

Suppose that it holds for infinitely many n. Dividing by X" and taking the limit as n — > °°, we deduce 
that P + y = 0. Then y is an eigenvector of S. If y ^ 0, then y is a rational eigenvector of 5, and X and 
At 1 are rational numbers that are algebraic integers. Hence, X = ±1, which is a contradiction. This 
implies that y = 0. Then it follows that Bx = (equivalently, = x). This shows that Mil holds for 
finitely many n only. Thus, the sequences T" and S",n> 1, are relatively jointly mixing. □ 

Lemma l2.27l implies the following special case of Coniecture l2.25l 

Proposition 2.28. Let T £ SL(2,Z) fee unipotent, and S G SL(2,Z) hyperbolic. Then for any mea- 
surable T> C T 2 , f/ze Z/m/f of p.{p n T"© nS"2)) as n — > 00 exists, and 

lim juf® nr© n5"©) >^(®) 3 - 

Moreover, the equality holds if and only iffj(CD) — 1 or 0. 

Proof. Let / be the characteristic function of the set © . Denote by fy and Ps the orthogonal projec- 
tions on the the spaces of T- and S-invariant functions respectively. Since S is ergodic, Pgf = p.{D). 
By Lemma l2.27l 



In the case when T and S are unipotent, Coniecture l2 . 251 seems to be open in general. We prove a 
partial result for sets of special form. For a function / g L°°(T d ), its Fourier coefficients are denoted 



(21) 



r T"x + r S"y + z = 




a 




Proposition 2.29. Let T,S € SL(2,Z) fee unipotent. 

(1) For any measurable D C T 2 , f«e //m/f o/ - ^('T' n T"© n 5"© ) as n 



°° exists. 
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(2) Suppose that TS ^ ST. Let A £ SL(2, Z) be such that A 1 TA is lower triangular unipotent. 
Then for every set of the form © —A{p\ x ©2) where ©1 and ©2 are measurable subsets 
ofT\ 

iim J u(«nr"©n5"©) >p(<D f. 

Moreover, the equality holds if and only ifp('D) — 1 or 0. 

Proof. We prove (1) in the case when T and S do not commute. (When T and S commute, they are 
powers of the same transformation, and the proof goes along the same lines as the proof below.) 

Let v and w be primitive integer vectors such that *Tv = v and 'Sw = w. We claim that for f,g,h£ 
C°°(T 2 ), 

(22) [j&)g(T%)h(S>%)d^^ £ f(-iv-jw)g(iv)h(jw) as « ^ 00. 



It follows from a standard argument that it is sufficient to check 1221 when /, g, h are characters of 
the form (0. 

Let / = % v , g = % , and h = % t for some JC,y,z € Z 2 . First, suppose that x = —iv — jw, y = iv, 
z = jw for some ij £ 1? . Then 

This implies Q2> in this case. 

Now we consider the case when jc, y, z are not of the above form. We need to show that the 
equality x + 'T"y + 'S"z = holds for finitely many n only. Suppose that it holds for infinitely many 
n. Write 'T = E + B and 'S = E + C where E is the identity matrix and B,C £ SL(2,Z) such that 
B 2 =C 2 = 0. Then 

x + 'T"y + 'S"z ={x + y + z)+n(By + Cz) = 0. 

holds for infinitely many n. This implies that x+y + z = and By = —Cz. Note that the columns 
of matrix B are multiples of the vector v, and the columns of C are multiples of w. If By ^ 0, v is 
multiple of w, and it follows that in some basis of R 2 both T and S are unipotent upper triangular. 
Then TS = ST, and this contradicts the initial assumption. Thus, By — Cz = 0. Equivalently, Ty = y 
and Sz — Z. Hence, x — —iv — jv, y — iv, and z — jw for some i,j £ Z. This is a contradiction. We 
have proved J22i . 

Replacing T by A~ l TA and S by A~ 1 SA, we reduce the problem to the case when T is lower 
triangular and © = ©1 x ©2- Then v = '(1,0). Let w = '(a,b). Let / be the characteristic function 
of the set ©, and f\ and fa be characteristic functions of the sets ©1 and ©2 respectively. Note that 
for s,t £%, f(s,t) = f\ (s)fa(t). To prove part (2), we need to show that 

£ f(-i-aj-bj)f(i,0)f(aj,bj)>v(v) 3 . 
Using the Plancherel formula and the fact that f\ = fa , we have 

£ /(- i - aj, -bj)f(i, 0)f(aj, bj) = £ ( £ /1 (-aj - i)fa (i) j / 2 (-67) £ (0)f(aj,bj) 



Z{fi)(-oj)f2{-bM*>2)f(aj,bj) 



= ^2)£I/iMI 2 IA(6j)I 2 

yez 

> K«2)|/i(0)| 2 |/ 2 (0)| 2 >/i(©) 3 . 
We are done. □ 
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Next, we investigate mixing properties of subgroups of SL(2,Z). 

Proposition 2.30. Let H be a subgroup of SL(2, Z). The action of H on T 2 is mixing if and only if 
H contains no nontrivial unipotent elements. 

Proof. If the action of H is mixing, then the action of every infinite subgroup of H is mixing, and 
consequently, H does not contain nontrivial unipotent elements. 

Conversely, suppose that the action of H is not mixing. By Lemma l2~2l there exists a sequence 
h„ G H, n > 1, and (x,y) S (Z 2 ) 2 — {(0,0)} such that 'h„x — —y for every n > 1 and h n — ► °°. Then 
%J 'h n x — x for every n > 1 . Thus, /z„/z^ 1 6 // is a nontrivial unipotent element for sufficiently large 
n. This proves the proposition. □ 

A subgroup of SL(2,Z) is called nonparabolic if it contains no nontrivial unipotent elements. 
Nonparabolic subgroups are of interest from the point of view of ergodic theory because they are 
precisely the groups that act in a mixing fashion on the torus T 2 . It follows from the pigeonhole 
principle that every subgroup of SL(2,Z) of finite index contains a nontrivial unipotent element. 
First examples of nonparabolic subgroups were constructed by B. H. Neumann in |N| (see also 
I Mag |). Any Neumann subgroup has the property that powers of a single unipotent element form a 
complete system of representatives of the cosets of this group. In particular, Neumann subgroups are 
maximal nonparabolic subgroups. There are examples of maximal nonparabolic subgroups that are 
not Neumann (see |T|, |BrLl|, |BrL2|). If F is a free normal subgroup of finite index in SL(2,Z) 
which is not equal to the commutant of SL(2,Z), then the commutant of F is nonparabolic (see 
|Mas|). 

Although there are large subgroups in SL(2,Z) (e.g. Neumann subgroup) whose actions on the 
torus T 2 are mixing, the following proposition shows that for nonabelian subgroups the higher order 
of mixing is impossible. 

Proposition 2.31. A nonabelian subgroup o/SL(2, Z) cannot be mixing of order 2. 

Proof. Let H be a nonabelian subgroup of SL(2,Z). Suppose that H is mixing of order 2. Take 
g,h e H such that gh ^ hg. Since H is mixing, g and h are hyperbolic. Note that if g 2 h = hg 2 , g 2 
and h can be both reduced to the diagonal form, and this would imply that g and h commute. Thus, 
g 2 h ^ hg 2 . Put hi = g~'hg', i = 1,2,3. Comparing eigenvalues, we deduce that /i; ^ —hj. Also, 
hi ^ hj for i 7^ j, since otherwise it would follow that g and h commute. Therefore, by Proposition 
12.101 the transformations hj and hj are jointly mixing for i ^ j. In particular, h~[ n h n j — > °° and h" — > °° 
as n — > oo. On the other hand, since h\, hi, h^ have the same eigenvalues, it follows from Proposition 
I2.10l (and its proof) that for some fi,fi,f3 € L°°(T 2 ), 

iim / fi(h%)f 2 m)hm)d^ fji®<% (h<m, [j&M- 

This implies that the sequences h\ n h" 2 and h\ n h!\ are not jointly mixing, which, in its turn, contradicts 
the assumption that the group H is mixing of order 2. □ 

More generally, a similar argument allows one to show that the standard action on the torus T d of 
a subgroup H of SL(rf,Z) which is not virtually abelian can not be mixing of order d. Another ap- 
proach to the proof of this fact can be found in | Bh | where it is utilized for derivation of isomorphism 
rigidity for the action of H. 

Remark 2.32. It should be noted that in contrast with the nonabelean situation, any mixing 7L d - 
action on T d is mixing of all orders (see | S , Corollary 27.7]). On the other hand, if an SL(d, Z)-action 
is a restriction of an ergodic measure preserving SL(d, R)-action, then by a theorem of S. Mozes (see 
|Moz|), it is mixing of all orders. 

We conclude by proving a result of Krengel-type |Kr|, which can be considered as a generaliza- 
tion of the fact that every ergodic automorphism of the torus has countable Lebesgue spectrum. 
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Proposition 2.33. Let H be a subgroup o/SL(2,Z) which acts in mixing fashion on T 2 . Then for 
every f G L 2 (T 2 ) and every E > 0, there exist /o G L 2 (T 2 ) and a subgroup Hq of finite index in H 
such that 

(23) / /o(^)/o(^ = 

Jt 2 

for every h G Hq, h e. 

Proof. First, we note that since H is mixing, for every (x,y) G (I?) 2 — {(0,0)} there is at most one h 
such that r hx = y. Indeed, if 'h\x = 'hjx for some /ii,/i2 G H, then hih^ 1 G H is a unipotent element. 
One can choose 

m 

with some a,- G C and Xj G Z 2 such that 1 1 / — fo \ \ 2 < £. We have 

Jt 2 Jt 2 "*> *j 

Let hij G H — {e} be the unique element such that 'hijXi — Xj (if such an element exists). Since 
SL(2,Z) is finitely approximable, the subgroup H is finitely approximable too. There exists a sub- 
group Hq of finite index in H such that ^ Hq for every i,j = 1, . . . ,m. Then d23t holds for every 
h G i/o> h^e. This proves the proposition. □ 
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